DOCOMIMS IBSDll 



ED 137 321 TB 006 111 

ADTHOB Sirois, Herman 1, i Iwanick^ BdMard F. 

TITLE Dalphi-^Diacrspaney S^aluations k Model for the 

Quality Control of Fedaral^ State ^ and Locally 

Mandated Programa* 
PUB DATE £lpr 77] 

HOIS 25p,l Paper presented at tt© annual Heeting of thf 

American iducational Eesearch issoGiation (61 st^ Mew 
lork^ New Xork, April 4*8^ 1977) 

SDBS PEICE MF-^$Q.83 HC-^f1*67 Plus Postage* 

DISCBIPIOES Educational Programsj ^E^aluation Metiiodsi Evaluation 

Needs; ForaatiTe Evaiuationi *flodelai Needs 
lesessment I ^Program Evaluation i SummatiTe 
Evaluation 

IDBNTIFIIES *Belphi technique 



ABSISACI . 

legally mandated educational programa often lack 
specificity and guidelines for such programs are often vague and 
subject to conaiderable variability in interpretation. This situation 
' presents perennial pi^blems for evaluators^ Few evaluation models 
have the flexibility for dealing vith this ambiguity and variability 
while at the same time achieving program evaluations which are both 
formative and summative. Xhe Belphi-Discrepancy Evaluation model is 
designed to assess and prescribe* Thm five phases of the model 
includei (1) establishing a standard by using the procedures of the 
Delphi Xechnigue/ (2) gathering data about programs baaed on the 
standard, (3) determining the discrepancy status of programs, (4) 
analyzing these discrepancies in relation to criteria and other 
programs in the population^ and (5) prescribing and remediating based 
on the anaiyaes of discrepancies. Bringing together successful 
techniques.- such as the Delphi Method, Discrepancy ivaluationr and th^ 
notions^of Euclidean Distance to bear on a particular evaluation 
problem should facilitate analyses and render daclsion-makera wit^ 
more valid information for formative and sufflfflative purposes, 
^Author) 



* Documents acguired by laxc include many informal unpublished * 

* materials not available fropa other sources. BEIC makes every effort * 

* to obtain the best copy available. Nevertheless, items of marginal * 

* reproducibility are often encountered and this affects the guality * 
^ of the microfiche and hardcopy reproductions EBIC makes available ^ 

* via the lEIC Document Eeproduction Service (EDIS) * EDBS is not * 

* responsible for the guality of the original document, fieproductions * 
^ supplied by EDIS are the best that can be made from the original. ^ 



EKLC 



\ 



mimx-mBcsmmcY evaluation: 

A MODIL FOR TEE QUALIOT COKraOL OF FlTOfflAL, 
STATE, imd LOCALLY MmQATm PROGRAMS 



U S. DEFARTM6NT OF HEAWtH, 
gDUCATlON & WELP&RE 
NATIONAL {NSTITUTE QW 
ipUCATiON 

TNIS DOCUMENT MAS BEEN REPSQ- 
DUCiD EXACTLY A% HfeCEIViU ^ROM 
THE PERION Q^GANiZATlDN ORiGfN- 
ATING n POrNTS OF ViEW OR OPINiONS 
STATED DO NOT NECESSARILY REPSi^ 
Sf Nf OFFICJAL NATJQNAL JNlTiTUTE OF 
eOUCATjON POSiTiON OR POLiCY 



Henaakn A* Sirals 
Idward F* Iwaniak 
University of Connect ieut 



Paper present ed at the annual oonventlon of the American Edueational Research 
Association* Nev York^ Aprils 1977* 



Delphi-DisarepanGY Evaluations 

A Model for the Quality Control of Federal, State 

and Looally Mandated Programs (EVA) 

HEHimN Ai, SIROIS EDWA^ P. IWMICKI 

University of Connecticut 

-ABSTRACT- ; 

Problem . I^egally mandated educational programs often lack 
spaciflcity and guidelinee for auch programs are often vague and 
subject to considerable variability in interpretation* This 
situation presents perrenial problems for evaluators* Few 
evaluation models have the flexibility for dealing witti this 
ambiguity and variability while at the same time achieving program 
evaluations which are both formative and emrmative^ 

Methodology ^ The Delphi-Dlsorepancy Evaluation model is 
designed to assess and prescribe. The five phases of the model 
include : (1) Establishing a standard by using the procedures of 
the Delphi Technique, (2) Gathering data about programs based on 
the standard, {3) Determining the discrepancy status of programs, 
(4) Analysing these discrepancies in relation to criteria and 
other programs in the population^ and (5) Prascribing and remediating 
based on the analyses of discrepancies. 

Conclusions , Bringing together successful techniques such as 
the Delphi Method, , Discrepancy Evaluation, and the notions of 
Euclidean Distance to bear on a particular ev 

facilitate analyses and render decision-makers with more valid 
information for formative and suinmative purposes, 
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The evaluation of federal, state, and local educational 
programs has beeome a large volume and high finance business. 
The evaluation industry has evolved around a growing cult of 
"experts," each passing the cult's initiation rites of de- 
veloping a new or modified evaluation model which promises 
to resolve the perennial evaluation problems of methodology, 
application and implenientation , relevance, impact, value, 
policy, and politics. 

Of the numerous evaiuation models in existence, such 
as Scriven's Formative-Summative model, his Goal-Free model, 
his Modus Operandi model, Stake's Countenance model, Stuf- 
flebeam and Guba's CIPP Ccontent, input, process, product) 
model, Provus' Discrepancy model, Rippey's Transactional 
model, Owens' Adversary model, Wergin's -Political model, 
and Cook's PERT (program evaluation and review technique), 
etc., almost all were designed to meet specific needs of the 
various programs to which they were initially applied. Much 
of the development (and progress) in educational evaluation 
appears to be a result of the failure of one model or another 
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of evaluation to meet the evaluation needs of a ppogram when 
applied inappropriately to that programs and the subsequent 
refinement and/or redevelopment of an evaluation model to 
meet the unique needs of the situation or program in question* 

A lesson we can derive from the history of educational 
program evaluation ^ to datej is that no one evaluation model 
can serve all evaluation needs. Rather ^ the evaluation needs 
of each particular program or situation must he analyzed and 
a program-specif ie evaluation plan must be developed ^ drawing 
from any and all existing sources of evaluation ffiodels^ to meet 
the information, feedbacks judgemental ^ and decision-making 
needs of the program under consideration, A Popham suggests^ 
educational evaluators should disregard Polonius' dictum: 
"Neither a borrower nor a lender be (Pophamp 1B7S^ p*H2) J' 

The evaluation design presented in this paper is a hybrid 
models, begged and borrowed from proven models in the areas of 
educational evaluation 5 educational planning and futuriBtica^ 
and special education needs assessment. This model was dev'e.Vbped 
to meet both the formative and summative evaluation demands of 
mandated educational programs where at least loosely defined 
guidelines or criteria exist..' 

The concepts of Delph-DiscrepanGy Evaluation is presented as 
a model for district-wide ^ state^wide^ and/or nation-wide monitor- 
ing and needs assessment of mandated educational prograins. The 
model, originally presented in May^ 1976 to the Connectieut State 
Department of Education by Sirois S Levin (1976)5 was designed to 
maet the state department's desires to CD monitor mandated LEA 
teacher evaluation programs in relation to state' guidelines ^ and 
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(2) diagnose spoaific LEA neada (inservicej assistance 3 etc.) 
in relation to developing teaciier evaluation program^ 

The model incorporates tha Delphi Technique for establish-- 
ing critical or criterion needs ^ and the notions of discrepancy 
evaluation for arriving at judgements regarding the remedial needs 
of local school districts. The modal is essentially a blend of 
the Delphi Planning Technique CWeaverj 1973) ^ the piscrepaney 
Evaluation model CProvuSj 1971) ^ and the notions of Euclidean 
Distance (from Gable 6 Glllungj 1976)^ and is appropriately named 
the "Delphi*-Discrepancy" model of program evaluation* 

The Delphi Flanning Technique 

The Delphi Planning Technique was or*iginally developed as the 
Delphi Forecasting Method by Olaf Helmer at the Rand Corporation 
^'think tank" during the mid-sixties (Helmer 3 1967). Cyphert & 
Gant (19 71) charactarize the Delphi Technique as a model or set 
of procedures for achieveing consensus. Weaver defines the pro** 
cess as "*-ian intuitive methodology for organizing and sharing 
^expert'" opinion (1973 5 P- 4U). 

In its ideal application ^ the Delphi Technique has been used 
to achieve a consensus of opinion amongst experts regarding the 
probability of the occurrance of certain events at some future 
time. The methodology has also been used to estimate the desir- 
ability of the occurance of such events. Redefining the Delphi 
Technique in terms of its applicability to the monitoring of 
state-mandated teacher evaluation 5 it might read' A technique 
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to achieve consensus amongst selected local school districts 5 
previously judged to either have or have the potential of having 
"good" programs^ "good", that is, in regards to the elements and 
activities that might comprise the "ideal" program, 

Weaver^B concise summary of the Delphi Technique (19735 p. 45) 
is paraphrased for the present context: Tj-pically, the proceduro 
includes a questionnaire ^ mailed to respondents who remain 
anonymous to one another. Respondents first generate several 
rather concise statements of criteria and/or activities that they 
consider important to the mandated educational program under con- 
sideration. In a second rounds respondents are provided with a 
coalated list of all respondents' criteria and are asked to 
prioritize the items on the list in terms of their importance to 
the mandated educational progr^, A third round provided each 
respondent with another coalation of itemB^ this time in rank 
order according to the averaging of the prioritization responses 
in round two* At this time, respondents are asked to prioritize 
the items once again either agreeing or disagreeing with the 
averaged list provided them. The final cofilation of the averaged 
prioritization of the items becomes thft "standard" against which 
comparisons are made. 

Discrepancy Evaluation 

Discrepancy Evaluation, as developed by Provus (1971)5 begins 
at the point of establishing a "standard". In program evaluation, 
this standard is usually considered to be the design of the program. 
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In the present application , the Delpiil Technique accomplishes this 
task. This design or standard is next compared to the actual initial 
installation of the prograin. If a discrepancy exists s it is cofr-ected 
before, the.. program is allowed to proeede. The third step in the 
discrepancy model is to compare the standard to the "process" which 
dsvelops and exists after the program has been installed. Again, 
discrepancies are noted and remediated. A fourth step in the model 
compares the products oi^ outputs of the program to tho^^ which w^pe 
hypothasized in the design or standard. A final stage involves 
cost--dasign discrepancies* 

As with the Delphi Techniques the application of Discrepancy 
analysis to the monitoring of mandated educational progi^ams involves 
analogous procedures and techniques. Once a "standard" has been 
developed (via the Delphi Technique)^ local school district education* 
al programs are compared to the appropriate standard (s) at approp^£^ 
ate stages of development, 

Delphi-DiscreDancy Evaluatio n 

Delphi-Discrapancy Evaluation has a dual purpose: to assess and 
to prescribe. The five phases of the Delphi-Discrepancy Evaluation 
model described below are designed to accomplish these two purposes. 

The phases of the Delphi-Discrepancy Evaluation model are de- 
scribed in conteKt below as they would apply to the evaluation of ^ 
specific mandated educational program^ i-e, state-mandated teache^* 
evaluation in Connecticut. 

In 1974^ the Connecticut State Legislature mandated teacher 
evaluation programs of all LEAS, The Connecticut State Department 
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of Education CGSDE) developed a set of eleven guidalines to which 
all such programs ^ would necessarily conform. The Educational Resourcas 
and Development Canter CERDC) at the University of Connecticut was 
contracted to conduct annual evaluations of state-wide teacher eval- 
uation program progress « 

The data from the ERDC evaluations represent the point of de- 
parture for the following descriptian of the Delphi-Discrepancy 
Evaluation Model as applied to state-mandated teacher evaluation in 
Connecticut, 

Phase I : Establiehing a Standard 

Phase I involves the use of the Delphi Technique for the pur- 
pose of generating a set of indicators or '^standards" which will 
form the basis for the discrepancy analysis and prescriptive-remedial 
phases of the Delphi-Discrepancy model. The sequence of tasks in 
phase I are itemized below i 

(1) Identification of an "Expert" S^ple: Using the prior 
CSDE/ERDC ratings of Connecticut local school district teacher eval- 
uation programs (Sheathelmp et* al,5 1975 1 Sirois^ et. al.^ 1976)^ 
determina whiah fifteen or twenty districts have been rated as hav- 
ing the "best" teacher evaluation programs. This group will be con- 
sidered the "expert" respondents in the Delphi phase of the Delphi- 
Discrepancy Evaluation process. 

(2) For each of the eleven state teacher evaluation guidelines ^ 
request of the respondents a detailed list (akin to brainstorming) 
of criteria 5 indicators s activities ^ etc. which they consider to be 
indicative of "good" teacher evaluatj^qn programs . 

C3) Coalate all responses Ci^e., within each of the eleven 
categories as defined by the eleven categories as defined by the 
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guidelines) * 

(4) Provide each respondent with this coalated lists asking 
each respondent to prioritize all items within each of the eleven 
guidelines . 

(5) Secure an average priority rating for each item and coalate 
all items again ^ rank-bordering them under each of the eleven guide* 
lines, 

(6) Provide each respondent with this coalatedj prioritized 
list. Ask each respondent to re-prioritize the lists either 
agreeing with the averaged priorities as they appear on the lists 
or changing them as opinion may dictate* 

(75 Perform a final coalation of the eleven lists of items ^ 
again ranked by average priority rating* (For conveniences only 
the top ten 5 fifteen ^ or twenty items might be retained for the 
final lists). This final list should take the form of the sample 
displayed in Figure 1* 

(8) The final form of the list(s) as presented in Figure 1 
will serve as a working checklist for the "standard"* This form 
should include three rating categories with values i (0) ^indicator 
not in operation at all, (1) -indicator in partial operations ^nd 
C2)=lndicator is in full operation Cor has occurred fully) in the 
district -s teacher evaluation program. 

(9) The indicators should be weighted according to their 
priority ranking. One method for such a weighting has been used 

in the sample in Figure 1 * The indicator ranked as top priority is 
given the greatest weight, in this case a value of 10* The second 
ranked indicator is assigned a value of and so forth. At a later 
stage in analysis, these values are .■multiplied by the ratings 
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PIGU^ 1? A SAmm PRIORITIZED UST 
~0P INDICATORS FOR GUIDEUCNE I* — 



GUlEaSLINE Is Each profasslonal shall oo. 
Qparativaly dtt#rmina with the eiralu- 
ator(e) tha objeotives upon which his 
or her evaluation shall be iDased. 



10# Written obj actives for the evalu^ 
atee are developed. 

9e Objectivaa are developed oootser- 

a1 ^ 
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• Objectives are stated in measure- 
abie ' terms . 








« A master list (or pool) of objec- 
tives is developed for the school 
systenii 















(ft, is or 2) which results from the data : gathering pha^^^ 

; Phase II i Determining the *^Status" of Local Edueational Programs - 
The checklists developed in phase I may be, used to gather data in 
a. number of ways. Each of these methods utilizes the notions of dis- , 
crepancy evaluation CProvuSs 1971) » In each case^ the indicators on thd 
bheeklists represent the optimum standardSj which the ratings secured 
from observations and/or self-'reports represent degrees of actual ' ; 

status of local teacher evaluation programs * 

-*Type A'* Discrepancy i In phase II of D^phi^Discrepancy Evaluation 
we are seeking to eliminate any occurrance of a "type A" discrepancy ^ 
and arrive at a collaboration as to the "status" of the teacher eval- 
uation program in question* Each of the conditions of data gathering i: 
requires collaboration from at least two sources - A type A discre- 
pancy is said to occur when any of the conditions of data gatherings 
described beloWs result in significantly different ratings on any of * - 
the checklists • 

At least three options or conditions of data gathering exist for 
the Delphi-Discrepancy evaluatori 

(1) On-Site Visitation/ Superintendent Report: In this ease ^ the 
superintendent completes the checklietCs) while a representative of 
the mandating agency CCSDE) conducts an on-site visitation^interview 
to complete a second such checklist . A "type A" discrepancy exists 
if the two assessments are significantly different* 

(2) Local^Reference Groups Method i The superintendent ^ a princi- 
pals* group 5 and one or more teachers ' groups complete separate . . 
checklists* A "type A" discrepancy exists if the euperintendent ; 
report differs significantly from either of the other two reports. 



(3) The Adversary Method: The superintendent appoints two 
parties or teams to oomplete separate cheaklists. One party is 
designated ae a critical adversary ^ while the other party repre« 
sents the superintendent. A type A discrepancy exists if the 
two reports are significantly different* 

A "significant" difference between groups cannot be determined ' 
statistically when each group consists of only one subject as is 
called for in the present design^ A somewhat arbitrary level of 
significance must^ therefore ^ be established. This fiat is easily 
accomplished: (1) Determine what is the highest possible "score" 
on the particular guideline checklist by summing the products of 
two Crating category 2^completion of indicator) times the weight- 
ings of each indicator (this total might be approximately 110 for 
any guideline in the present application). (2) Determine the 
signifioant discrepancy value by taking 1S% of this total (in the 
present example the significant discrepancy value would be 17)^ 
(3) Consider any type A discrepancy which has a value greater than 
the significant discrepancy value to be a significant type A dis- 
crepancy. 

Resolving a Type A Biscrepancy i If a type A discrepancy 
exists 5 it must be resolved prior to testing for "type B" discrepancy 
in phase III of the Delphi^Discrepancy model. Three methods of re^ 
solution of type A discrepancies are suggested here. 

(1) First 3 for insignificant type A discrepancies ^ the various 
forms of the checklists may simply be averaged for a consensus 
rating. . . 
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(2) A second method attempts to rasolve a signi^ieant type 
A discrepancy by returning the forms to the respondents and ask- 
ing them to get together to complete a eonsansus form of the 
checklist. Failing consensus at this point 5 the third method 

of discrepancy resolution is instituted, 

(3) The third method also attempts to resolve a significant 
type A discrepancy* This discrepancy is referred to the mandating 
agency (CSDE) for further verif ication^ on-site visitation 5 etc, 

in an attempt to gather an accurate assessment of the status of the 
teacher evaluation program in question. 

Whatever method is used to resolve a type A discrepancy ^ the 
resolution becomes the "status" of the particular teacher evaluation 
_ program under consideration* 

Phase III ^ "Type B" Discrepancy - Standard ^ Status 

The "status" of any particular teacher evaluation program is 
represented by the ratings Con the eleven checklists) assigned to 
that program after any type A discrepancy has been resolved during 
pjiase II of the Delphi^Discrepancy Evaluation process. 

"Type B" Discrepancy :i , Once the question of a type A discrepancy 
has been resolved and the "status "of a particular teacher evaluation 
prograjn has been determined 5 we may then analyze the nature of any 
"type B" discrepancy which may exist. This is done; by comparing the 
"status" of teacher evaluation programs to the "standard". As de- 
fined in Delphi-DiBcrepancy Evaluation, a "type B" discrepancy is 
not considered to be an absolute measure** Rather 3 it is considered . 
to be the relative discrepancy of a district's teacher evaluation 
program when compared to the "standard" in relation to all other 

15 . . 
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district teacher evaluation programs • 

A variation iri the application of the Euclidean Distance formula 
as suggested by Gable S Gillung (1976) will provide an index or 
number 3 the magnitude of which indicates the similarity between the 
status" and the "standard" (a low value indicating a high similarity) 
By generating such an index for each school district teacher evalua^ 
tion program 3 comparisons can be made across districts on the basis 
of such variables as per pupil expense ^ size^ unban-suburban-rural ^ 
mean income ^ eto.. More important for the immediate purpose of the 
CSDE (the mandating agency) ^ such an index will enable a ranking of 
school districts according to the Euclidean Distance values (by 
guideline) thus facilitating the identification of the remedial 
needs of local school districts in relation to their teacher evalua- 
tion programs . ^ 



A^g - the actual status rating for indicator 
number i of guideline number g* 

I^g = the ideal (standard) rating for indicator 
number i of guideline number g, 

2 • indicates that the value (A^ - l^)- is 
computed for each indicator and these 
values are sununed under each guideline. 

Ng ^ the number of items or indicators under 
guideline number g, 

=A number of Euclidean Distance values may be generated. .First , 

and most appropriate ^ eleven such values can be obtained for each 




Where : ^ g 



the Euclidean Distance value for 
guideline niimber g. 
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local teacher evaluation progFam Cone for each of the eleven guide- 
lines)* Other such values may be obtained by considering groups of 
guidelines together for comparison to other distrieta or to other 
groups of guidelines 5 or simply totalling the eleven EuGlidean Dis-- 
tanca values for gross comparisons between districts. (It should be 
noted that in such variations ^ the values li and Ng would vary). 

Any resulting Euclidean Distance value is not interpetable in 
any direct manner. Ratherg the value and power of Euclidean Distance 
values lias in the fact that they may be compared directly to other . 
Euclidean Distanca values of the mamB population* It can be con-* 
eluded 5 for example ^ that a teacher evaluation progrant in district 
X5 with a distance value of 5 on guideline number is in need of 
greater remedial assistance in relation to such components as setting 
objectives and agreeing on objectives for evaOtuation cycles than in 
district which may have received a distance value of 2 on guide- 
line number I« 

Phase IV : Analysis of Discrepancies 

Given the standardized quality of the Euclidean Distance values 
and the reliability and utility of their comparison to 6ther Euclidean 
Distanca values from the same population ^ four different analytical 
procedures are suggested here to assist the CSDE in CD assessing 
statewide status ^ and (2) developing remedial plans in relation to 
local teacher evaluation programs , The first three of these pro- 
cedures "block on" or compare the various local teacher evaluation 
programs to each other in relative terms . The fourth procedure 
blocks on or compares the eleven state teacher evaluation guide- 
lines to each other, - ^ 



Cl) Rank Order r Based on the Euclidaan Distanae values ^ a 
rank ordering of the local teachar evaluation prDgr^s should he 
performed. The higher the Euolidean Distance valuer the more a 
particular teacher evaluation program departs from the "standard" ^ 
and is in need of greater attention and/or assistance from the - 
CSDl, Such a rank order ^ if performed for each of the eleven state 
teacher evaluation guidelines ^ would provide considerable diagnostic 
information for the CSDE in its efforts to assist and remediate . 
Also^ such rank orderings would yield a direct measure of the ''status' 
and "compliance" of local teacher evaluation programs in relation to 
the CSDE guidelines. - 

^2) Clustering : Based on the rank ordering of the teacher 
evaluation programs conducted in procedure number one above ^ clusters 
of school districts within each guideline may be identified (either 
visually or by a computer sub-program) which are simil«* in terms 
of their stages of development ^ level of "compliance" to guidelines ^„ 
and level of attention and/or remediation required by their teacher 
evaluation programs . 

(3) Item Analysis : As an optional procedi^et for a more detailed 
diagnottic analysis^ the CSDE may wish to cluster local teacher 
evaluation programs based on their Euclidean Distance values on 
each item or indicator within each of the eleven guidelines. Such 
analysis would not only identify the particular guidelines which 
are presenting problems for local districts, but would provide the 
CSDE with information regarding the particular indicators which are 
contributing to such problems. C It. should be noted that changes in 
certain computational procedures ehould be considered when analyzing 



data based on indicators s e*g. the weightings of the indiGators 
would neGeesarily be ignored and not entered into any computations) 
CM^) Guideline Block Analysis i PE'ocedures onm^ two^ and three 
are designed to compare the various local teacher evaluation pro-* 
grams to each othare Procedure number four compares 5 on a state-^ 
wide basis 5 each of the eleven state teacher evaluation guidelines 
with each other* The value of such a compariGon is that those 
guidelines which are causing problems for many school districts are 
readily identified. This^ of course 5 would have definite impli* 
cations for the focus of state-wide teacher evaluation workshops ^ 
and other large scale CSDE teacher evaluation activity. 

Note I The four procedures suggested above do not exhaust the 
analytical proaaibilities nor do they resolve all the problems of 
assessment of status and identification of necessary remediation* 
The Delphi-Discrepancy a valuator should be prepared to conduct 
further analysis tailored to specific information needs of the 
mandating agency and, of the local school districts* A combination 
of procedures numbers two and fourp for example^ would produce a 
cross sort table indicating the priority needs of both local dis- 
tricts and specific guidelines. Such an analysis is displayed in 
Figure 3 below and is discussed in phase V of this presentation ■ of 
the Delphi- Discrepancy Evaluation model. 
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se Vj Planning for' Assist^oe i Prescription »^ mnd Remediation ^ 
The Use of Discrepancy AnmlymXB . " : ; 

The CSDE (or mandating agenGy) will undoubtedly wish to provide ■ 

assistance to local teacher evaluation programs ^ within the pe%«ennial 

conetraints of available funds. The results of the Delphi-Diserepancy 

Evaluation procedures can assist the mandating agency in planning ;i 

workshops and other more direct in-service activities which can be 



designed to meet both the particular and the conmion needs of the ^^^^^^^ 
various school districts in an efficient manner. A discussion of 'A;:, 
Figure 3 should demonstrate how the analytical procedures suggested ;V 
in phase IV of the Delphic-Discrepancy model can achieve this purpose V ": 
Please note that the discussion 'below is based on the purely hypo- 
thetical data from Figure 3. / ^ ' , V ; 

Referring to Figure 3 and recalling that a larger Euclidean r 
Distance value indicates a need for attention and/or assistance in / ' 
relation to the teacher evaluation program in question ^ we notice 
that certain cells in the cross sort have obviously greater values r 
than do other cells. Cell III-3 (guideline #111^ district #3) ^ for 
exMiple has a noticably larger value than any other district under 
guideline III, / ' ! 

A look at guideline IV reveals that all districts are having 
problems with this guideline ; This would suggest to the' mandating ^ 
agency that a one-topic j state-wide workshop in relation to guide- 
line IV (job de script ions ) ^might be in order as a remedial technique* 

Considering the values under guideline I, however v such a state* C 
wide workshop dealing with setting objectives (guideline I) would not 
be advised. A small group workshop ^ however s or some other form of 
assistance to school districts numbers 3 and S does seem warranted 
in relation to the high values of these two districts on guideline I*; / 
:'. ■ :■■ =■ . \- 21 - . 



Finally j it appears from the values in Figure that district 
number 3 is in need of some special attention and assistances since 
it has high values under every guideline^ General workshops for 
district number 3 might prove useless at this point* Some intense 
consultationj financial assistances etc, from the mandating agency 
might be in order for distric number 3 . 

The above discussion ^ hopefully , has made clear the remedial 
applications of the Delphi^Discrepaney Evaluation model. Without 
the actual data resulting from a Gomplete application of the Delphi^ 
Discrepancy method, specific eKamples and suggestions for in-service 
and remedial assistance , of course^ cannot be made^ The point is, 
simply 5 that the results of the Delphi-Discrepancy analysis can have 
definite implications for remediationi specific remedial needs of 
local distriats can be identified via the Delphi-Discrepancy methods 
The effective and creative use of the model can render such assist- ^ 
ance both more efficient and considerably more effective . 



Conclusion 

The methodology of the Delphi-Discrepancy Evaluation model as 
presented in this paper appears to be of significant value in achiev-- 
ing the dual purpose of (1) aseessing the status of local educational 
programs in relation to guidelines established by mandating agencies , 
and (2) identifying the specific remedial needs of such educational 
programs in a diagnostic manner. Although the techniques that make 
up the Delphi^-DiSGrepancy model (i,evf the Delphi Technique^ . 



DiscrepanGiy Evaluationf and the notione of Euclidean Dtstance) have 
not yet bean applied or tasted together? each has proven quite Valu-^ 
able and reliable in r.solated appliQations of the various techniques^ 
Bringing all three techniques together to bear on an evaluation pro- 
blem such as that posed by the Connecticut State Teacher* : ^ 
Evaluation Act CP. A, 74'-278) should facilitate the analysis of the 
pr^oblam and render decision-makers sets of valid and reliable data 
on which to^base judfenients. 

The uee of the Delphi^DiscrepanQy Evaluation model la not li^iitedl 
to the applications as presented herein* Other pro grams ^ state ^V^^oal 
and federal would benefit from appropriately designed compreheneive 
evaluation models such as the Delphi-pis cbepancy model* Again 
noted earlier 5 an evaluation model such as the- Delphi-DisGrepanoy 
technique should not be siijiply superimposed on a pi?Qgram. Rathet*^ 
the evaluation needs of the program should be analysed and the 
evaluation model should be modified and adapted to suit program 
needs. 
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